Skip to content

fix(test): stop rebuilding env/keys per iteration in repo-name validation tests - #6880

Merged
JSONbored merged 1 commit into
mainfrom
worktree-github-labels-timeout-flake
Jul 17, 2026
Merged

fix(test): stop rebuilding env/keys per iteration in repo-name validation tests#6880
JSONbored merged 1 commit into
mainfrom
worktree-github-labels-timeout-flake

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • test/unit/github-labels.test.ts's "rejects invalid repository names before making GitHub calls" and test/unit/github-pr-actions.test.ts's "validates the repo name before any GitHub call" each called createTestEnv() up to 9 times, and generated a fresh RSA-2048 key (generateRsaPrivateKeyPem(), real synchronous CPU work) up to 4 times -- once per malformed-name case in a loop.
  • ensurePullRequestLabel/closePullRequest reject on parseRepoFullName()/splitRepo() before ever touching env (src/github/labels.ts, src/github/pr-actions.ts), so none of that setup was needed more than once per env-shape (plain vs. keyed).
  • These were reported as timing out alongside three already-merged sibling fixes for the same general class of problem (real subprocess/DB setup work exceeding vitest's timeout under heavy concurrent load): test/unit/ai-summaries.test.ts (fix(test): stop ai-summaries forbidden-word sweep timing out under load #6874), test/unit/agent-sdk-driver.test.ts (fix(test): reduce and widen the agent-sdk-driver real-git test timeout #6871), and the #5132 miner clone/worktree suite (fix(test): widen timeouts on real-git miner clone/worktree tests #6869).
  • This pair didn't reproduce under realistic load, though: 3x isolated runs and 3x runs under simulated CPU contention (16 processes oversubscribing a 12-core machine) all passed cleanly, with neither target test ever appearing among the slowest tests in its file. The original failure was traced to two full ~18k-test suites running concurrently on one machine (two worktrees' npm run test:ci backgrounded at the same time) -- a self-inflicted, unrealistic doubling of load, not a normal-load risk.
  • Fixing the confirmed redundant setup anyway since it's the exact same safe, zero-behavior-change pattern already proven on the three sibling PRs, and it measurably speeds up the suite regardless: ~20% faster on both files together under sustained load (44.7s/45.4s/40.9s fixed vs. 56.0s/56.4s/54.8s baseline, 3 runs each, same continued CPU-contention conditions). No timeout was widened -- there was nothing to widen; this is purely eliminating unnecessary setup.

Scope

  • The PR title follows type(scope): short summary Conventional Commit format.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked a currently open issue this PR resolves -- not applicable; this is a maintainer-authored test-reliability fix, not a contributor PR under the linked-issue policy.

Validation

  • git diff --check
  • npm run actionlint
  • npm run typecheck
  • npm run test:coverage locally -- test-only change to test/**, which Codecov does not measure, so there is no patch-coverage obligation.
  • npm run test:workers
  • npm run build:mcp
  • npm run test:mcp-pack
  • npm run ui:openapi:check
  • npm run ui:lint
  • npm run ui:typecheck
  • npm run ui:build
  • npm audit --audit-level=moderate
  • New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries -- not applicable; no production code changed, and both files' full 58 tests still pass unchanged.

Additional validation beyond the standard checklist: both target tests run 3x in isolation and 3x under simulated heavy CPU contention with zero failures (see timing data in Summary). A full local npm run test:ci run (single suite, not concurrent with anything else) shows both github-labels.test.ts and github-pr-actions.test.ts fully clean.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics. (Not applicable -- test-only change.)
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests. (Not applicable -- no such changes.)
  • API/OpenAPI/MCP behavior is updated and tested where needed. (Not applicable -- no such changes.)
  • UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks. (Not applicable -- no UI changes.)
  • Visible UI changes include a UI Evidence section below. (Not applicable -- no visible/UI changes.)
  • Public docs/changelogs are updated where needed. (Not applicable.)

UI Evidence

Not applicable -- test-only change, no UI/frontend/docs surface touched.

Notes

…tion tests

test/unit/github-labels.test.ts's "rejects invalid repository names before making
GitHub calls" and test/unit/github-pr-actions.test.ts's "validates the repo name
before any GitHub call" each called createTestEnv() up to 9 times and generated a
fresh RSA-2048 key (generateRsaPrivateKeyPem(), real synchronous CPU work) up to 4
times -- once per malformed-name case in a loop. ensurePullRequestLabel/
closePullRequest reject on parseRepoFullName()/splitRepo() before ever touching
`env` (src/github/labels.ts, src/github/pr-actions.ts), so none of that setup was
needed more than once per env-shape.

Reported as a timeout flake alongside three already-fixed sibling tests
(test/unit/ai-summaries.test.ts, test/unit/agent-sdk-driver.test.ts, the #5132
miner clone/worktree suite), but this one didn't reproduce under realistic
single-suite-equivalent load (3x clean runs under simulated CPU contention) --
the earlier failure was traced to two full test suites running concurrently on
one machine, not a normal-load risk. Fixing the confirmed redundant setup anyway
since it's the same safe, zero-behavior-change pattern already proven on the
sibling PRs: measured ~20% faster on both files together under sustained load
(44.7s/45.4s/40.9s vs a 56.0s/56.4s/54.8s baseline, 3 runs each), all 58 tests
still pass, and a full local npm run test:ci pass shows both files fully clean.
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@JSONbored JSONbored self-assigned this Jul 17, 2026
@JSONbored
JSONbored merged commit 865d1b1 into main Jul 17, 2026
12 of 13 checks passed
@JSONbored
JSONbored deleted the worktree-github-labels-timeout-flake branch July 17, 2026 10:10
@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 17, 2026
@codecov

codecov Bot commented Jul 17, 2026

Copy link
Copy Markdown

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
17777 1 17776 13
View the top 1 failed test(s) by shortest run time
test/unit/release-selfhost-prerelease.test.ts > release-selfhost.yml "Resolve version" step (#1937) > accepts a stable semver tag and marks it non-prerelease
Stack Traces | 0.117s run time
AssertionError: expected { v: '0.1.0', tag: 'orb-v0.1.0', …(2) } to match object { v: '0.1.0', tag: 'orb-v0.1.0', …(2) }

- Expected
+ Received

  {
    "prerelease": "false",
-   "release": "gittensory-orb@0.1.0",
+   "release": "loopover-orb@0.1.0",
    "tag": "orb-v0.1.0",
    "v": "0.1.0",
  }

 ❯ test/unit/release-selfhost-prerelease.test.ts:58:23

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant